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ABSTRACT 

Developed by the Advanced Systems Development 
Division of International Business Machines (IBM) in consultation’ 
with two professors, the Educational and Career Exploration System 
(ECES) includes information on 400 occupations and 300 post-high 
school majors. Designed for the high school student and for use with 
a computer terminal, the system includes charts which summarize and 
compare information about the student and his explorations. To 
evaluate the program, data were gathered on an experimental group and 
a control group of 10th graders, and also from counselors and parents 
of the students. Analysis of the data revealed that: (1) While 

students tended to concentrate on occupations during their first 
sitting at the ECES terminal, they used their second sitting to 
explore both occupational and educational opportunities, (2) Parents 
of users were generally pleased with the effects of bhe system and 
stated that they were more involved in the educational and vocational 
planning of their children than they had been previously, and (3) 
Counselors had positive reactions to the system and were optimistic 
about its effect on students. Despite these positive results, 
however, the anticipated gains in vocational maturity were not 
realized during this first year of the field test. (Author/JS) 
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INTRODUCTION 



Developed several years ago at the Advanced Systems Development Division 
of International Eusiness Machines (IBM), in consultation with Professors 
Donald E. Super and Roger A. Myers of Teachers College, Columbia University, 
the Educational and Career Exploration System (ECES) had its first field 
trial and evaluation at Montclair High School in New Jersey in the Spring of 
1969. The results of that field trial were reported in detail in ar unpub- 
lished monograph (Thompson et al.« 1970), and summarized in a paper by Bohn 
et al . , read at AERA in 1970, and in several other places in the professional 
literature. ' After initial trial ECES was revised and expanded by IBM in 
preparation for a county-wide field trial in Genesee County (Flint), Michigan. 
This report presents some of the results of the first phase of that project. 

In its present configuration ECES contains three sections: Occupations 
(400 occupations, representing many fields and levels), majors (300 post-high 
school, college, and other training programs), and charts , which summarize 
and compare information about the student and his explorations. These three 
sections are available to the student in an interactive mode. The terminal 
which the student uses consists of a film image display unit which presents 
relatively static information and questions and a typewriter which presents 
relatively individualized information to the student. In addition a fourth 
section of ECES, a post-high school program locator called College Finder, 



is available without the use of a terminal. 
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BACKGROUND AND METHOD 

. ' i 

The field test and evaluation of ECES in Genesee County was conceived 
as a five-year project. The primary purpose during the first year was to 
develop a sound program of use patterns and to develop and refine instru- 
ments. The second and third years were planned as the evaluation phase, 
followed by analysis and follow-up during the final years. Since this is a 
report of only the first year's activities, the evaluative data must be 
viewed as tentative and preliminary. 

Rationale for the Evaluation Plan 

During the past 20 years those most concerned with the study of occupa- 
tional choice, career guidnce, and the evaluation of vocational guidance 
procedures and programs have substituted the concept of career or vocational 
development for that of occupational choice, and the developmental criterion 
of vocational maturity for static criteria such as having an occupational 
choice or having a realistic occupational choice. The history of this change 
cannot be described in detail here, but some of the major issues, the major 
studies, and the important findings to date need to be identified to make 
clear the basis for the evaluation procedures selected for refinement in the 
first year of this project. 

The developmental approach in vocational counseling seems to have had 
its origins in the 1930's and 1940's through both sociological and psycho- 
logical studies in the USA and abroad (Davidson and Anderson, 1937; Buehler, 
1933; Miller and Form, 1951; Ginzberg et al., 1951). The Career Pattern 
Study (Super, 1954, 1955; Super et al. , 1957; Super and Overstreet, 1960; 
Super, Kowalski, and Gotkin, 1967) demonstrated that the conventional ob- 
jectives and criteria of vocational guidance were invalid and irrelevant in 



O 

ERIC 



3 



3 . 






the 9th grade. While it is true that most 9th graders can and will, when 
asked to do so, report an "occupational choice", changes in educational and 
occupational plans are frequent during the high school years and consistency 
of choice or of field of choice is not related to other traits of known im- 
portance in 9th graders. Furthermore, even measures of realism or wisdom of 
occupational choice in the 9th grade are unrelated to occupational or career 
success or satisfaction at age 25. The findings of the Career Pattern Study 
have been confirmed by several independent investigations. The Career 
Development Study (Gribbons and Lohnes, 1968, 1969) essentially confirmed 
in 8th, 10th, and 12th grades the findings reported for the Career Pattern 
Study. In studying occupational plans and the outcomes of vocational guid- 
ance in several Wisconsin high schools, Rothney (1958) confirmed the in- 
stability and virtual irrelevance of early adolescent choices, as did 
Flanagan and others in Project Talent (Flanagan and Cooley, 1966; Cooley and 
Lohnes, 1968). 

Appropriate objectives and criteria for 9th grades were shown, by the 
Career Pattern Study, to involve planfulness and time perspective (the 
tendency to look ahead and plan for anticipated situations), having and 
seeking needed information, and knowing what kinds of information are likely 
to be needed. These have been characterized as indices of vocational 
maturity , indices which show increases as adolescents go through high school 
and enter college and the world of work. A number of measures of vocational 
maturity have been developed, most during the past 10 years. It was Crites 
(1965) who first developed a practical measure, the Attitude Scale of the 
Vocational Development Inventory . He is currently working on a Competence 
Scale to complete the VDI, but, at this time, only the global score on the 
Attitude Scale is ready for use. The National Assessment Project has developed 
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items for assessing vocational maturity, but these are not now available to 
other researchers. Westbrook (Westbrook and Clary, 1967; Westbrook and 
Cunningham, 1970) also has developed an objective instrument, the Cognitive 
Vocational Maturity Test, which seeks to measure only the intellectual or 
informational, as contrasted with the affective or motivational aspects of 
vocational maturity. 

Vocational maturity was measured in this project by the Career Develop- 
ment Inventory (CDI), developed specifically for the project by Super, Bohn, 
Forrest, Jordaan, Lindeman, and Thompson (1971). The objective was to create 
an independent, virtually self -administering inventory which would assess as 
many of the relevant vocational maturity factors as possible in one school 
period. Drawing upon the work of Super, Crites, Gribbons and Lohnes, and 
Westbrook, 11 dimensions of vocational maturity were initially identified. 

Items were written and scales developed to measure these dimensions. After 
pilot work involving factor analysis and item analysis, the scales were used 
in revised form during the 1970-71 academic year in the Genesee County study. 
Analysis of pretest results from 10th graders during Spring, 1971, led to the 
development of the scoring system upon which the results given in this report 
are based. The 11 dimensions initially identified were reduced to three, 
namely. Planning Orientation (Scale A), Resources for Exploration (Scale B), 
and Decision Making and Information (Scale C). Details of the development 
of these three vocational maturity scales of the CDI , as well as evidence of 
their validity and reliability are given in Forrest (1971), It suffices here 
to state that the three scale reliabilities (retest) range from .56 to .76, 
and that intercorrelations with the Crites, Gribbons and Lohnes, and Westbrook 
vocational maturity scales, and with intelligence, socioeconomic status, and 
school achievement, show that one CDI scale (C) is cognitive while two (A and 
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B) are affective, the last two being virtually independent of intelligence 
and socioeconomic status. 

Objectives of the Study 

Based on the findings of previous resealch outlined above, and keeping 
in mind the primary purpose of the first phase of the study, the objectives 
for the first year were formulated as follows: 

1. Determination of the effects of the system on student vocational 
behaviors as evidenced through tests of vocational development, 
acquisition of occupational and dec is ion -making knowledge, appro- 
priate information -seeking activities, contacts with counselors, and 
evidence of planning activities. 

2. Determination of the effects of the system on counselor attitudes 
toward vocational counseling, on changes in the quality of their 
contacts with student users, and on their attitudes toward the 
system as an aid in counseling. 

3. Determination of the attitudes of student users and their parents 
tox^ard the system and its reported effects on their educational 
and vocational planning. 

4. Determination of optimum usage patterns for the second and third 
years of the field trial. 

5. Refinement of the evaluation objectives, methods, instruments, and 
sampling procedures. 



Subjects 

The students participating in the field test and evaluation were 10th 
graders of the 25 public high schools of Genesee County, Michigan. To be 
considered eligible for inclusion in the study a student had to have on file 
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a grade -point -average from the previous year and complete scores on the O hio 
Vocational Interest Survey (OVIS) and the Vocational Planning Inventory (VPI). 
Both instruments were used in producing a "search strategy" to help students 
use ECES. The 25 schools were paired on the basis of size, socioeconomic 
level, location, ethnic composition, number of counselors, and dropout rate. 

One member of each pair was randomly designated experimental, the other con- 
trol. The experimental schools had 3,201 eligible sophomores, while the con- 
trols had 2,386. 

The data gathered on these students included the following: 

1. The Career Development Inventory, administered to both experiments Is 
and controls in December, 1970, prior to exposure of the experimentals 
to the system, and again in May, 1971. Subscales comprising the three 
scales (A, B, and C) of the CPI are listed in Table 1. 

2. Student Reaction Form . This questionnaire was administered immediately 
after use of the system to assess ECES users' immediate reactions to 
system format, ease of use, clarity of presentation, and to the 
general worth of the system. 

3. S tudent Weekly Activities Report (SWAR). This brief checklist dealt 
with the activities of reading about, looking into, and discussing 
educational and occupational topics. It was administered weekly for 
10 weeks to a sample of 250 experimentals and 250 controls who com- 
prised the intensive study group. 

4. Student Questionnaire . This form was administered in May, 1971, to 
a sample of ECES users to determine general reactions to the system 
and to assess its perceived effects on and contributions to the 
students' educational and vocational plans. 
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The 114 counselors in the 25 schools constitute the other major group 
of subjects in the evaluation. Data from counselors included the following: 

1. Survey of Guidance Attitudes . This was a 27 -item multiple -choice in- 
strument administered to all counselors in December, 1970 and in May, 
1971 to assess, before and after the experimental period, their be- 
lief in the value of the vocational counseling function in guidance. 

2. Counselor Reaction Questionnaire . This instrument, given to all 
counselors in the experimental schools, was designed to assess the 
counselors' experience with ECES, the administration of the program, 
and interaction with student ECES users in counseling situations. 

At the end of the experimental period a Parent Questionnaire was sent to 
the parents of a sample of experimental students to determine the effects of 
system usage on the interactions of parents with students in regard to educa- 
tional and vocational planning activities. It also assessed parent attitudes 
toward the system. 

Procedures 

To obtain information on usage patterns when ECES is available on a 
voluntary basis to all sophomores, two terminals were placed in a large inner- 
city Flint school, and one in a suburban school. Students from the other 11 
experimental schools were transported by bus to nine terminals located in a 
single room in the District headquarters building (GISD). The latter groups 
of experimental students received two two -period block sessions, one week 
apart. No optional sessions were available. With this arrangement only about 
60% of the eligible students from these 11 schools could be accommodated on 
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RESULTS 



Use Patterns 



Because optional use of the system at the GISD location was not possible, 
there was little opportunity to determine optimal usage patterns based on the 
present study. Combining all the students who used ECES, regardless of where 
and when they used it, produced the following distribution of hours of use. 

No. of Hours Used % of Users 



6 or more 
5 
4 
3 
2 
1 



4 
1 

44 

5 

38 

8 



It is clear from this distribution that the modal number of hours of use was 
a function of the scheduling of experimental students at GISD. It is also 
clear that many of the students, for one reason or another, did not return to 
GISD for the second scheduled session. These data are much different from 
those obtained in the Montclair field trial (Thompson e_t a_l. , 1970) where the 
mean number of hours of usage by 10th graders was 6.7. At Montclair, however, 
students could use the terminals a 3 often as they wished. 

Concerning the use of the two major parts of the system, the data show 
that during their first two-hour session approximately two-thirds of the 
users explored occupations while one-third explored majors. In the second 
session nearly all users explored occupations and two-thirds investigated 
majors. Apparently moGt students utilize both of these aspects of the system. 



Student Reactions Immediately Following Use 

Responses to the Student Reaction Forms were overwhelmingly favorable, 
as indicated in Table 2, where only the overall reactions to the system are 
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recorded. The great majority of students indicated highly favorable reactions 
to specific features of the system as well. The physical features of the sys- 
tem were seen as easy to use. The system was judged to explain ideas, occupa- 
tions, and majors well. The charts were considered helpful, and the system 
was seen as being helpful with educational and occupational planning. After 
the second session more students wanted a longer period at the terminals, 
and more of them wanted to use it more often, suggesting that the value of the 
system became more apparent after greater familiarity was achieved. Although 
nearly all students gave favorable responses, the non-urban students were 
more enthusiastic than the urban students. Females were somewhat more en- 
thusiastic than males, indicating they found it easier to use than did the 
males, and that they felt they received more help in educational and occupa- 
tional planning. 

Student and Parent Follow-up Reactions 

Responses to the Student Questionnaire (SQ) and the Parent Questionnaire 
(PQ) were obtained late in the school year when all experimental students had 
completed their use of the system. These instruments were distributed to 340 
students who had used the system for too sessions or more and to their parents. 
Although only about one-fourth of the students and parents returned usable 
questionnaires, responses of those who did was highly favorable. The students 
indicated that they felt more definite about plans after using ECES, that 
they could profit more from talks with counselors after using the system, and 
that the system was sufficiently personal, but was not making decisions for 
them. Most parents reported having discussed ECES with their children and 
estimated that because of ECES they had become more involved in their child's 
educational and vocational planning. In general, the parents' responses were 
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highly favorable and indicated a desire for their children's earlier intro- 
duction to the system. 

Effects on Students: Vocational Maturity 

Pretest and posttest data on the three scales of the CPI constitute the 
data on which the following results are based. Two types of analyses were 
done to compare experimental and control groups of 10th grade students: 

1) those using sex and type of community as independent variables, and 2) 
those using race and sex as independent variables (restricted to schools in 
which both white and black students were represented). 

No significant differences between subgroups of students were found on 
Scale A, Planning Orientation, in either type of analysis. Significant 
differences were found, however, on Scales B and C; the means of subgroups in 
the analyses of these scales are shown in Tables 3 through 5. We may summarize 
the results of these analyses of covariance as follows: 

1. Treatment — Statistically significant differences between ECES users 
and non-users were found only among student*? from urban schools where 
there were a substantial number of black students enrolled. ECES 
users tended to show slightly greater improvement than non-users in 
terms of quality of potential and used occupational resources, as 
measured by the CPI . The difference, however, was small, being 
approximately one-fourth of a standard deviation. While it is true 
that this finding suggests a positive effect of the limited 1970-71 
usage of ECES, it is not clear that the effect of two two-hour sessions 
at the terminals is of sufficient magnitude to enhance significantly 
the vocational maturity of students. 

2. Sex - -Statistically significant differences between males and females 
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were found only on CPI Scale C, Decision Making and Information. 
Females improved slightly more than males on this scale during the 
period of study, but the difference was so small that it is of little 
practical or theoretical significance. 

3. Race --Statistically significant differences were also found between 
black and white students on CPI Scales B, Resources for Exploration, 
and C, Decision Making and Information. On Scale B, the difference 
was in favor of white students, and on Scale C in favor of blacks. 
Differences were small in both cases and of doubtful practical signi- 
ficance. 

4. Location — Statistically significant differences were found on CPI 
Scale C, Decision Making and Information, between rural, suburban, 
and urban schools participating in the study. Rural schools tended 
to show slightly greater improvement during the study period, but, 
again, differences were small. 

The results presented here show that with the pattern of use in the 
Genesee County schools in Spring, 1971, ECES had little practical effect on 
career development. Non-users increased in vocational maturity at approxi- 
mately the same rate as ECES users. Whether positive ECES effects would be 
observed under different, perhaps expanded, usage patterns and whether longer- 
term effects will be observed as the study continues are not known at this 
time. 

Effects on Students: Information-seeking Behavior 

The Student Weekly Activities Report (SWAR) was administered to 10th 
graders in the three schools comprising the Intensive Study Group. It was 
administered by classroom teachers on a routine weekly basis from January 
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through May, 1971. Many of the forms showed careless markings which rendered 
interpretation of the data impossible. Such forms were not used in the analyses. 

For purposes of analysis samples of approximately 100 control students and 
60 experimental students were drawn. The SWAR yielded four scores based on 
whether the activities concerned educational or vocational pursuits and whether 
they took place in or out of school. No significant differences between ex- 
perimentals and controls nor between males and females were found on any of 
the four scales. Furthermore, no significant changes were found in experi- 
mental students before and after using ECES, although some scores appeared to 
change in the expected direction, i.e., in the direction of a greater propor- 
tion of educationally and vocationally related activities. 

Counselors 1 Attitudes toward Vocational Counseling 

One of the objectives of the study was the assessment of counselors' 
attitudes toward the vocational guidance aspects of the counselor's role. The 
expectation was that such attitudes might be affected by the presence of ECES 
as an addition to the array of guidance media. 

The Survey of Guidance Attitudes was mailed to all 114 counselors in the 
County at the beginning and again at the end of the school year. Eighty-one 
percent of the counselors returned completed SGA's at the beginning and 68% 
at the end. Responses indicated a widely-shared positive attitude toward 
vocational guidance activities on both pre- and posttest with no significant 
shift over time. Comparisons made among counselors from experimental vs. con- 
trol schools, among those who reported different titles and job duties, and 
among those who reported spending differing amounts of time in counseling, re- 
vealed no dignificant differences on either pretest or posttest. 
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Counselors * Attitudes toward ECES 

The Counselor Reaction Questionnaire (CRQ) was constructed on the basis 
of interviews with counselors in the experimental schools. In May, 1971, the 
CRQ was sent to all experimental counselors. Twenty-nine of the 40 experi- 
mental counselors returned usable forms. 

Counselors' reactions to the effects of ECES on students were generally 
positive and supportive of the system. Exceptions were to the statements that 
ECES greatly reduced anxieties about occupational choice and that certain of 
the ECES messages proved discouraging to the student. Only about half of the 
counselors agreed with these statements, the other half disagreeing. Although 
students were seen as benefitting from ECES, the counselors were not uncritical 
of some of the specifics of the system. 

Counselors' views of ECES hardware, scripts and mechanics were also 
generally positive, although opinion was divided on the ideal length of a 
student's interaction with the terminal and on the advisability of making ECES 
available to 9th graders. Concerning the effects of ECES on themselves, most 
of the counselors agreed that it had created more work for them, though this 
reaction is probably not due to the system itself, but to the experimental 
situation in which they were placed. A large number agreed that ECES had 
caused them to do some reading they would not otherwise have done, and many 
stated that they had been stimulated to do some independent local research 
on ECES users. 

About half of the counselors responding felt that they were seeing types 
of students they had not previously seen. It was also significant that about 
half thought they were working with students at a "much higher level of problem 
solving" following ECES use. 
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SUMMARY 



The first year of the field trial of ECES provided opportunities to ex- 
plore various means of collecting data and suggested ways in which more use- 
ful data could be collected in the future. In general, it was learned that 
10th grade students could use the system and enjoyed doing so. Though they 
tended to concentrate on occupations during their first sitting at the ECES 
terminal, they used their second sittings to explore both occupational and 
educational opportunities. Parents of users were generally pleased with the 
effects of the system and stated that they were more involved in the educa- 
tional and vocational planning of their youngsters than they had been pre- 
viously. Counselors had positive reactions to the system and were optimistic 
about its effects on students. 

Despite the positive reactions of students, parents and counselors, the 
anticipated gains in vocational maturity was not realized. There was, how- 
ever, some reason to suspect that more exposure to the system might have re- 
sulted in greater changes in planning orientation, the use of resources for 
exploration, and the quality of decision making and the amount of occupational 
information possessed. 

The field trial continues. Improved instrumentation and data collection 
techniques should provide more definitive answers in the years that follow. 
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Table 1. Subscales comprising the A, B, and C Scales 
of the Career Development Inventory. 



Scal e 

A: Planning 

Orientation 



B: Resources for 

Exploration 



C: Decision Making 

and Information 



S ubscale No. of Items 

Specificity of planning 14 

Self-estimated amount of 

occupational information 11 

Definiteness of plans 1 

Concern with choice _7 

Total, Scale A: 33 

Quality of potential 

resources 15 

Quality of used resources 15 

Total, Scale B: 30 

Knowledge of decision- 
making principles 12 

Measured occupational 

information 18 

Total, Scale C: 30 



Table 2. Students' overall reactions to ECES 
immediately following use. 



Question: What is your overall reaction to the ECES system? 



Response 



After First Use 



After Second Use 



I like it very much. 328 
I like it somewhat. 54 
I dislike it somewhat. 4 
I dislike it very much. 0 
No response. 34 



53 

11 

0 

0 

356 
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Table 3., Pretest and posttest means and adjusted posttest means and 
standard errors of experimental and control groups on CDI 
Scale C--by sex and type of community. 



Type of 






Pretest 


Posttest 


Adjusted 


Posttest 


Community Treatment 


Sex 


N 


Mean 3 


Mean 3 




Meanb> c 


St. Error 


E 


M 


63 


16.6 


16.8 




17.1 


.43 




F 


72 


16.9 


17.6 




17.7 


.40 


Urban 
















C 


M 


159 


17.1 


17.3 




17.2 


.27 




F 


149 


17.4 


17.9 




17.6 


.28 


E 


M 


86 


17.4 


16.5 




16.3 


.37 




F 


118 


17.6 


18.2 




17.9 


.31 


Suburban 
















C 


M 


240 


17.0 


17.1 




17.1 


.22 




F 


300 


17.0 


17.7 




17.7 


.20 


E 


M 


39 


16.7 


17.5 




17.8 


.55 




F 


33 


16.0 


17.5 




18.2 


.60 


Rural 
















C 


M 


76 


16.7 


17.8 




18.1 


.39 




F 


77 


17.0 


18.3 




18.3 


.39 


Standard deviations 


for subgroups 


are not 


given. 


The 


average within-groups 


standard deviation was 4.7. 














b 

Standard deviations 


for subgroups 


are not given. 


The 


average within-groups 


standard deviation was 3.4. 














c 

Main effect of Sex: 


F=8.57, 


1 and 


1359 d. 


f. (pC.oi) 






Main effect of Type 


of Community: 


F-~5 . 28 , 


2 and 1399 


d.f. (p.< 


.025) 
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Table 4. Pretest and posttest means and adjusted posttest means and 
standard errors of experimental and control groups on 
Vocational Maturity Scale B — by race and sex. 



Treatment Sex 
M 



E 



Race 



Pretest Posttest 
N Mean 8 Mean 8 



Adjusted Post test 
Meant *, c St. Error 



B 

W 



54 241.3 245.6 

18 258.4 266.7 



243.0 4.87 

252.7 8.46 



F B 52 262.1 260.3 243.8 5.03 

W 25 246.4 269.4 263.4 7.16 



C 



M B 146 227.8 231.6 238.0 2.98 

W 30 232.6 248.7 251.9 6.53 



F B 140 233.0 240.3 243.2 3.03 

W 35 239.8 237.2 235.6 6.05 



Standard deviations for subgroups are not given. The average within-groups 
standard deviation was 46.3. 

k 

Standard deviations for subgroups are not given. The average within-groups 
standard deviation was 36.1. 

c , 

Main effect of Treatment: F=4.23, 1 and 491 d.f. (p\.05) 

Main effect of Race: P=4.75, 1 and 491 d.f. (p^.05) 



Table 5. 
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Pretest and posttest means and adjusted posttest means and 
standard errors of experimental and control groups on CDI 
Scale C — by race and sex. 











Pretest 


Posttest 


Adj usted 


Posttest 


Treatment 


Sex 


Race 


N 


Mean 8 


Mean 8 


Meanb > c 


St. Error 




M 


B 


49 


17.3 


17.8 


17.7 


.49 






W 


14 


14.4 


13.2 


15.2 


.93 


E 


















F 


B 


53 


17.7 


18.4 


18.0 


.48 






W 


19 


14.9 


15.4 


17.0 


.80 



M 


B 


132 


17.8 


17.8 


17.3 


.30 




W 


27 


13.9 


14.6 


17.0 


.68 


C 














F 


B 


124 


18.0 


18.5 


17.8 


.31 




W 


25 


14.2 


14.9 


16.9 


.70 



Standard deviations for subgroups are not given. The average within-groups 
standard deviation was 4.7. 
b 

Standard deviations for subgroups are not given. The average within-groups 
standard deviation was 3.4. 

Main effect of Race: F=6.66, 1 and 434 d.f. (p^.025) 
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